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ABSTRACT 
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Ability Tests 2. Half-year results from which grades were derived were also 
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school as the unit of analysis, and student as the unit of analysis were 
supported, and further analysis using the SA metric showed that teachers were 
not gender biased in their grading of males and females relative to their 
IQs. The SA analysis revealed teachers extreme grading bias with respect to 
the student's age (95.8%) and a bias toward students' attitudes. The study 
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Introduction and initial exploration of ‘Situated Attainment’: 
Differences in ranked IQs and in-class attainment. 

Tony Bastick 

University of the West Indies 



Abstract 

If all else were equal within a classroom we might expect that children’s intelligence would be a 
strong determinant of their attainment. However, there are often other factors that come into play that 
considerably reduce the correlations between I.Q. and Attainment so that, in a particular classroom 
situation, a student’s actual academic attainment may be more or less that we might have expected 
based on his or her I.Q. scores. ‘Situated Attainment’ is a within-class concept that allows us to 
explore why teachers award some students higher or lower grades than their intelligences might 
warrant. 

A student’s Situated Attainment (SA) is operationally defined as the difference in ranks between 
the grade awarded to the student by the teacher and the student’s intelligence quotient. Situated 
Attainment is the difference between Grade rank and IQ rank within the class: SA=RG-RIQ. When 
students attain over their IQ expectation, their SA is positive. When students attain below their IQ 
expectation, their SA is negative. It was postulated that teachers’ and students’ value judgements are 
made mainly relative to class norms so that using the class as the unit of analysis (CUA) would reveal 
relationships that were less obvious when the individual was used as the unit of analysis (IUA) or the 
school was used as the unit of analysis (SUA). 

For this initial exploration of the concept, nine classes of Grade 9 students were selected, one 
from each of nine schools (n=3 19). These students, 127 males and 192 females of average age 14 
years 5 months, responded to an attitude questionnaire and a sub-set of culturally and 
psychometrically appropriate I.Q. questions from the standardized General Ability Tests 2 (GAT -2). 
The students’ half-year test results, from which their Grades are derived, were also recorded. The I.Q. 
scores were calculated and within each class, the students’ I.Q. scores and their grade attainments 
were ranked so that their SAs could be calculated. 

The theoretical assumptions of using CUA, SUA and IUA were supported, and further analysis 
using the SA metric showed that teachers were not gender biased in their grading of males and 
females relative to their IQs. The SA analysis, however, revealed teachers’ extreme grading bias with 
respect to student’s age (95.8%) and a bias towards student’s attitudes. It revealed the negative effect 
of continued underachievement on students’ self-esteem and showed that 62.5% of student 
dissatisfaction with low grades was due to underachievement rather than lack of ability. 



Situational factors that affect grades awarded by teachers 

Teachers’ grading practices vary greatly and, particularly when incorporating alternative assessments, are 
likely to involve more subjective value judgments (Frary, Cross, & Weber, 1993; Plake, & Impara, 1993). 
Teachers’ grading is a mostly private practice that Brookhart, refers to as “hodgepodge grade of attitude, effort, 
and achievement” (1991, p. 36). For example, the literature reports that teachers’ grading is affected by the 
student’s temperament (Holbrook, 1982), how independent, tidy and attractive a student is (Clifford, 1975; 
Ross, & Salvia, 1975), and the teacher’s personal relationship with the student (Pedulla, Airasian, & Madaus, 
1980; Doherty, & Conolly, 1985). Interestingly, it is not necessarily the attractive students who are over-graded. 
For example, Sparacino and Hansell (1979) found a negative correlation between physical attractiveness and 
Grade Point Average. 
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Because grades are so dependent on teachers’ values, researchers have chosen to use more ‘objective’ 
standardized measures for comparisons across classes and schools. Wentzel (1993) noted that grades reflect the 
social-emotional context of the classroom, such as teacher preference. In her 1993 study “Does Being Good 
Make the Grade? Social Behavior and Academic Competence in Middle School” Wentzel found that pro-social 
behaviour was a strong predictor of grades, but not standardized scores. Similarly, Schaefer and McDermott 
(1999) found that grades were significantly related to social behavior whereas standardized test scores related 
significantly only to IQ scores. Bennett, Gottesman, Rock, and Cerullo (1993), indicated that grades were 
influenced by gender and teacher perceptions of behavior. In a longitudinal study (Feldhusen, Thurston, & 
Benning (1970) with 384 least ‘socially approved’ third and sixth graders, positive social skills were found to 
correlate with grades over a 5 year period. Malecki (1998) also found that ratings of students’ social skills 
predicted academic achievement in the following term. 

Teachers’ grade evaluations may be influenced by many situational factors. For example, Yarborough and 
Johnson (1980) found that girls’ superior language and spelling achievement, coupled with their affective 
advantages, could be mistaken for superior reading skills. This is consistent with Hartley, (1982) who also found 
that teachers rated girls’s reading higher than boys’. This teacher gender bias towards girls has been found even 
when girls and boys have the same reading ability (Ross, & Jackson, 1991). 

There have been many and varied conformations of Rosenthal’s and Jacobson’s classic 1968 study 
showing how teachers’ perceptions of children influenced the their judgements (Babad, 1993; Brophy, 1983; 
Jussim, 1986, 1989; Weinstein, 1993). Race, ethnicity and SES are such factors that have been found to 
influence teachers’ judgement of children (Baron, Tom, & Cooper, 1985; Hall, Howe, Merkel, & Lederman 
1986). In a 1983 metastudy of 16 research results, Dusek and Joseph concluded that race and social class were 
major influences on teachers’ judgements. This was confirmed in an experiment by McCombs and Gay (1988) 
who manipulated information on students’ race, class and IQ to influence teachers’ (n=80) evaluations of 
pictures of a White child and an Hispanic child. 

The many situational factors that affect the grades awarded by a teacher makes such grades more suitable 
for comparison with other in-class grades awarded by the same teacher and less appropriate for wider 
comparison. 

Arguments for the ‘class’ as the unit of analysis 

If all else were equal, we might expect that students’ IQs were good predictors of their in-class grades 
(Lassiter, 1995; Poteat, Wuensch, & Gregg, 1988; Rodriguez, Prewitt, & Joseph, 1990). Interestingly, many 
measurement courses use concocted IQ and Grade data, or data from an academically homogeneous groups, for 
correlation exercises that illustrate the expectation that IQ and Grades are correlated. However, although there is 
a large literature on the use of standardized measures to predict future grades, such as the use of IQ and SAT 
scores to predict College grades (Bridgeman, Mccamley-Jenkins, & Ervin, 2000; Carvajal, & Pauls, 1995), there 
is comparatively little literature reporting significant correlations between individuals’ standardized measures 
and in-class grades across multi-school populations. In practice, situational variables can intervene to enhance 
or degrade performance, so that the grade percentage attained on a particular assignment may be higher or lower 
than that predicted by academic ability alone (Boulon-Diaz, 1992; Cuppens, 1967; Konarzewski, 1993; 
Sternberg, 1996). It is of some concern to educationists to identify situational variables that might act 
systematically to reduce grade scores of individuals or groups to levels below those justified by their abilities. 
‘Situated Attainment’ is a within-class metric that describes the difference between expected and actual within- 
class performance. It may be used to identify cases where situational variables may have intervened to influence 
performance scores, positively or negatively, and it may be used to explore the nature of these influences. 

Children and teachers naturally norm their educational value judgements in the context of their class, as 
their extensive time and involvement within their class environment can be expected to have a dominant 
influence on their evaluative experiences. For example, for socio-pedagogic reasons of motivation, self-esteem 
and accountability a teacher of a class students who are very below the academic average is unlikely to award 
O gnment percentage grades to the whole class that are correspondingly very low. Similarly, a teacher of a 
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class of students who are very above academic average is unlikely to award assignment grades to the whole 
class that are correspondingly very high. Hence, by taking the individual as the unit of analysis we would not 
expect to find that standardised academic ability and teacher awarded grades would correlate highly across 
school populations of varied academic standards. However, taking the class as the unit of analysis we would 
expect to find a correlation between students’ relative academic standing within the class and the grades 
awarded to students by the teacher of the same class. An early study of this kind was attempted by Krueger in 
1939 and subsequent within-class studies have found strong associations between standardized measures of 
academic attainment and grades (Fisher, 1995). 

Following this logic, it may be expected that within-class metrics could be more useful for exploring some 
socio-pedagogic issues than using the individual or school as the unit of analysis. ‘Situated Attainment’ (SA) is 
such a class-based metric. 

Situated Attainment is the difference between grade rank and IQ rank within the class. It can have many 
possible uses. A positive SA indicates the student’s grade was higher than would be predicted by their relative 
IQ. Negative SAs indicate under grading or under performance. Using SA we can separately identify under 
performance and underachievement. These words are defined here as: Under performance results in a standard 
below some group norm comparison (e.g. below average), and underachievement results in a standard below 
that predicted by the student’s academic ability. Where a grade is given for a short term assignment, a negative 
SA can be used as feedback to show that the student can be expected to produce better work. If a grade 
represents a combined mark over some longer period of time, such as an end of year grade, the student’s 
consistently lower than expected achievement is likely to have become evident to his or her peers. Such 
negative SA may then indicate serious underachievement which may be associated with peer related social, 
personal or behavioural problems (Goergi, 1972). 

Operational definition of Situational Attainment 

Situated attainment (SA) is the difference between how a student is ranked in-class by IQ and by 
assessment Grade (the grade awarded for work done, e.g. as a percentage, not year Grade as in k-12). SA=RG- 
RIQ where SA is Situated Attainment, RG is in-class Ranking by Grade, and RIQ is in-class ranking by 
Intelligence Quotient 

For ease of interpretation, reverse rankings are used, the smallest number being given the rank of 1, so that 
the higher the IQ or Grade then higher is the student’s rank. Table 1 illustrates calculations of the metric for an 
artificial data set for a class of 10 students. 

Table 1. Data illustrating the calculation of Situated Attainment 



Student 


Grade% 


IQ 


RG 


- RIQ 


= 


SA 




1 


40 


116 


3 


7 


= 


-4 


Undergraded 


2 


20 


104 


1 


4 


= 


-3 


Undergraded 


3 


65 


108 


7 


5 


= 


2 


Overgraded 


4 


62 


91 


4 


- 3 


= 


1 


Overgraded 


5 


83 


84 


10 


1 


= 


9 


Overgraded 


6 


63 


125 


6 


- 10 


= 


-4 


Undergraded 


7 


68 


117 


8 


- 8 


= 


0 


As Expected 


8 


30 


124 


2 


- 9 


= 


-7 


Undergraded 


9 


62 


84 


4 


1 


= 


3 


Overgraded 


10 


69 


115 


9 


- 6 


= 


3 


Overgraded 



Key 

IQ Intelligence Quotient 
RQ Rank by Grade 
RIQ Rank by IQ 
SA Situated Attainment 
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Although Table 1 illustrates an artificial example, we can see that positive values of SA indicate that the 
student has been over graded in relation to the prediction expected from their IQ. Similarly, negative values 
indicate under grading. It is proposed that we can use the SA metric to more sensitively explore variables that 
may contribute to over and under grading within the classroom, such as age, gender and personality differences. 

This study investigated assumptions and uses of Situated Attainment with a sample (n=319) of Jamaican 
adolescents. 

Method 

Nine Jamaican urban secondary schools were chosen at random for this study. Permissions were obtained 
for one 9"’ grade intact class in each of these schools to take part in the study and for the half-year grade 
percentages of these students to be used in the study. The subjects were 319 Jamaican adolescents, 127 boys and 
192 girls, with a median age of 14 years 3 months. The number of students in each class ranged from 30 to 40 
with a median of 36. It was considered that students’ scores in Mathematics and English would best represent 
their academic performance across school subjects. Hence, the average of each student’s percentage half-year 
grades in Mathematics and English was used as the Grade measure. 

The subjects completed a sub-set of verbal, spatial and logical IQ items chosen from the standardized 
General Ability Tests 2 (GAT-2) as being culturally appropriate and of fitting difficulty levels for the age of the 
sample. In addition, subjects were also given instructions to rate, from 0-9, the four self-esteem questions 
shown in Figure 1 . 



Figure 1 . Instructions and self-esteem questions. 

The following statements are about you. Rate each putting the best number from 0 to 9 in the box to show 
how much you agree that it is true of you. 

0 means ' not true of me ’ 1-3 means only a little true of me ' 4-5 means 'mostly true of me ’ 

6-8 means ‘very much true of me ’ 9 means ‘totally true of me 

Q 1 □ Today I am very happy 

Q2 dll feel I’m not achieving as much as my class mates in my schoolwork. 

Q3 dU I often wish I look like someone else. 

The following question is a NOT 0-9 rating. Read the question carefully and answer accurately. 

Q4 How many times last week did any of your friends avoid you? 



Analyses and results 

The subjects’ IQ scores, together with their half-term grade averages and responses to the self-esteem 
questions were coded for analysis. In addition, all scores were recorded as in-class ranks with the lowest scores 
being awarded the rank of 1 . The Situated Attainment for each student was calculated by subjecting the 
student’s within-class IQ rank from his or her within-class Grade rank. 

The normality of the SA metric is illustrated in Figure 2 and shown by the distribution parameters in Table 
2 and the following single sample fit statistic: Kolmogorov-Smirnov Z test showed that the SA distribution was 
close to normal Sig p=0.975 (2-tailed) 
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Figure 2: Normal distribution of Situated Attainment 




Situated Attainment 

Table 2: Distribution parameters of Situated Attainment 



SA Distribution Parameters 



N 


321.000 


Mean 


0.000 


Median 


0.000 


Mode 


-4.500 


Skewness 


-0.081 


Std. Error of Skewness 


0.136 


Kurtosis 


-0.308 


Std. Error of Kurtosis 


0.271 


Range 


67.000 


Minimum 


-35.500 


Maximum 


31.500 
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Testing the assumptions of Situated Attainment 

It was postulated that no significant association would be found between Grades and IQs using (a) the 
individual as the unit of analysis (IUA) or using (b) the school as the unit of analysis (SUA). It was postulated 
however, that a statistically significant association would be found between Grades and IQs when (c) the class 
was used as the unit of analysis (CUA). 

Assumption (a), that no association would be found using the IUA, was tested using the correlation of IQ 
with Grade for the whole sample. The correlation of IQ with Grade was not significant at r=0.077 (p=0.169, 
n=321) as postulated. 
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Assumption (b), that no association would be found using the SUA, was tested by comparing the mean 
grades of the schools with the highest and lowest mean IQ scores. First, an ANOVA was used to show that the 
schools differed significantly in their mean IQ scores. Table 3 shows this ANOVA and Figure 3 shows a plot of 
the mean IQs of each school. 



Table 3 ANOVA showing significantly different mean IQs between the 9 sample schools 



IQ 



IQ 


Sum of 
Squares 


df 


Mean Square 


F 


Sig. 


Between Groups 


739.006 


8 


92.376 


31.016 


.000 


Within Groups 


929.243 


312 


2.978 






Total 


1668.249 


320 









Figure 3: Mean IQ scores of subjects in each school 




School number 



The four schools with the highest mean IQs were grouped (5, 2, 4, 3) into a ‘High IQ’ group and the four 
schools with the lowest mean IQs were grouped (1,7, 8, 9) into a ‘Low IQ’ group. A t-test showed that the 
difference between mean IQs of the two school groups was very large, Table 4. However, Table 5 shows that the 
mean percentage grades of the two school groups was very close with no significant difference as postulated. 



Table 4: Schools’ analysis: Large significant differences between mean IQs of two school groups 



Variable 



Number 

of Cases Mean SD SE of Mean 



IQ 

Low IQ 
High IQ 

Levene' s 



147 4.4762 1.881 

142 7.2958 1.953 

Mean Difference = -2.8196 
Test for Equality of Variances: F= 



. 155 
. 164 

.014 P= .905 




Variances t-value 
Equal -12.50 
Unequal -12.49 



t-test for Equality of Means 95% 

df 2-Tail Sig SE of Diff Cl for Diff 

287 .000 .226 (-3.264, -2.376) 

285.52 .000 .226 (-3.264, -2.375) 
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Table 5: Schools analysis: No significant difference between the mean grade percentages of high and low 

IQ schools 



Variable 



Number 

of Cases Mean SD SE of Mean 



GRADE 
Low IQ 
High IQ 

Levene' s Test 



147 62.9434 
142 63.9544 
Mean Difference = 



10.113 .834 

10.870 .912 

- 1.0110 



for Equality of Variances: F= 1.022 P= 



.3i3 



t-test for Equality of Means 95% 

Variances t-value df 2-Tail Sig SE of Diff Cl for Diff 

Equal -.82 287 .413 1.234 (-3.441, 1.419) 

Unequal -.82 283.77 .414 1.236 (-3.444, 1.422) 



These results supported assumption (b) that no association between Grades and IQ would be found using 
the SUA. 

Assumption (c), that an association would be shown between Grades and IQ using the CUA, was tested by 
correlating students’ in-class IQ ranks (RIQ) with the in-class ranks of their grades (RG). The correlation of RIQ 
with RG was statistically significant at r=0.201** (p<0.0005, n=321), which supported the assumption. 

Uses of Situated Attainment 
Gender bias 

We might be interested to know if there are gender differences in how teachers award grades in relation to 
IQ. Are males, or females systemically under graded or over graded relative to their academic ability? A t-test 
showed that there was no significant differences between how the groups were graded and the means of both 
groups was very near to zero (-0.0940 for males and 0.0703 for females) showing that, overall, grading was 
equal and unbiased for both sexes. 

Age bias 

We might be interested to know if teachers are influenced by the age spread in their classes when they are 
grading. The age spread in the whole sample is greater than the age spread in any one class so, as expected, IUA 
showed IQ increased significantly with Age (r=0. 155*, p=0.008, n=290). However, perhaps unexpectedly, IUA 
also showed that grade levels significantly reduced with Age (r=-0.280**, p<0.0005, n=290), that is, there is a 
tendency for teachers in the whole sample to award lower grade marks to older students. So do teachers have 
expectations of older students that are too high? Using the CUA we find that the correlation between rank Age 
(RA) and RG is significant at r=-0.140* (p=0.17, n=290), indicating that older students are receiving lower 
grades in-class to a lesser extent than the IUA revealed. However, the correlation between RA and SA was 
significant at r=-0.143* (p=0.015, n=290). This shows that teachers are grading older students in their classes 
below the level warranted by their ability. Further more, this under grading accounts for (0. 140/0143) 2 = 95.8% 
of the under performance of older students. 

Social concomitants of under performance and under achievement 

Q 1 CH Today I am very happy 

Does a student’s serious or flippant attitude affect the grades a teacher awards? Using the Individual as the 
Unit of Analysis (IUA) there was no significant coloration found between self-reported happiness and Grades 
(r=-0.035, p=0.552, n=295). However, using the CUA we found happy students had a significantly lower SA, 
with a significant correlation between ranked scores on question 1 (RQ1) and SA of r=-0.146 (p=0.012, 
n=295). This means that unhappy students had higher SAs indicating that they were over graded. Why is this? 
q fbe teachers assumed happy students were more flippant, students who should have tried harder. Maybe 
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teachers interpreted unhappiness as a ‘serious’ attitude to the work, or maybe teachers wanted to ‘cheer up’ sad 
students by over-rewarding them for the work they did. 

021 I I feel I’m not achieving as much as my class mates in my schoolwork. 

The Individual Unit of Analysis (IUA) showed students who get low grades tended to be very dissatisfied 
with their performance. The correlation between Q2 and Grade was highly significant at -0.255** (p<0.0005, 
n=320). As expected, this effect was stronger within the class, because the question is relative to “class mates’. 
This was shown by the correlation of ranked answers to Q2 (RQ2) with RG of r= -0.301** (p<0.0005, n=320). 
However, the significant correlation of RQ2 with SA of r= -0.243** (p<0.0005,n=320) indicated that most of 
these students [(.243/.301) 2 = 65.2%] were disappointed in their low grades, not because they had a low 
potential, but because they had been under graded. 

Q3 d] I often wish I look like someone else. 

Q4 □ How many times last week did any of your friends avoid you? 

There were significant CUA correlations between the in-class ratings of the responses to these questions , 
RQ3 and RQ4, respectively, with SA. The correlation of RQ3 with SA was r= -0.157** (p=0.005, n=318 ) and 
the correlation of RQ4 with SA was r= -0. 1 74** (p=0.002, n309). 

The grades used in this study were half-year grades, so that this under grading represented an on-going 
situation of underachievement for these students. These correlations indicate that this long-term under 
achievement was accompanied by low self-esteem responses of wishing to look like someone else and by 
experiences of being avoided by ones friends. 

Summary' 

This study postulated that within-class metrics were more sensitive measures of teacher and student value 
issues than analyses across schools or comparisons of individuals across varied samples. The reason proposed 
for this is that teacher and student evaluations are very likely to be normed against their class experiences 
because of the disproportionate time and involvement teachers and students spend with their class environment. 
This reason was supported by reviewing studies showing teachers’ values influenced grades more than 
standardized scores. The assumption was empirically supported by showing the lack of association between in- 
class grades and a standardized measure of intelligence when using the individual and the school a the unit of 
analyses, compared to significant associations found when analysing IQs and grades ranked within each class. 

‘Situational attainment’ (SA) as a within-class metric, was suggested by a review of literature showing that 
situational influences, including teachers’ values, can enhance or degrade a student’s performance as predicted 
by standardised scores. SA is the difference in a students within-class Grade rank and their IQ ranking. A 
positive SA indicates over grading or over achievement whereas a negative SA indicates under grading or 
underachievement. Thus ‘underachievement’ is achieving less that one’s potential and is different from under 
performance, which is a norm group comparison, such as being below average. 

SA analyses on the Jamaican sample showed that teachers were not gender biased in their grading, nor did 
they over or under grade mails or females. Teachers were, however, highly biased in their grading against older 
students. SA analysis showed that older students tended to under perform and 95.8% of this under performance 
was due to Jamaican teachers’ higher expectation of older students over and above their ability. 

SA analyses also showed that teachers undergrade happy students, perhaps mistaking their happiness for 
flippancy or interpreting a ‘serious’ attitude as more fitting academic - or teachers may want to compensate for 
students unhappiness by over grading their work. 

SA analysis showed that 65.2 % of students reporting lower grades than their peers were being under 
-graded relative to their potential as indicated by their IQ. 

SA analysis also demonstrated the negative effect long-term underachievement has on students’ self- 
esteem. 

It should be noted that standardized within-class metrics would probably yield results with greater affect 
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sizes that ranked metrics, because of the greater variance accounted for by standardized measurers. However, 

SA was defined as a ranked metric to' make it more usable to teachers, as teachers can more readily rank their 
students than norm their scores. 

It would appear from the promising results in this study that the SA, and other within-class metrics, may 
offer more discriminating analyses of the wide value data, such as grades and attitude surveys, obtained from 
teachers and their students. 
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